1. Field of the Invention
The present invention relates to a composing system or apparatus, in the case of conducting simultaneous multi-image reproduction of a plurality of moving image (picture) data for one image or picture, using a method of encoding (coding) each of the moving image data and a composing method of producing a multi-image combined moving image corresponding to an arbitrary combination of the moving image data.
2. Description of the Prior Art
As exemplified by Japanese Unexamined Patent Publication No. 7-298263, there has been known a method whereby a synthetic code string obtained in a manner that a frame of a moving image is divided into a plurality of areas and code strings each produced by the encoding of each of the areas are connected to each other becomes identical in syntax with a code string created by encoding the image without the division into the areas.
FIG. 51 is a block diagram showing an arrangement accomplishing this method, and an operation thereof will be described hereinbelow with reference to this illustration.
First of all, a moving image inputted is divided through an image divider 61 into a plurality of areas in units of frame. At this time, the image divider 61 determines a division method on the basis of a image division control signal from a division control unit 75. In this instance, one frame is vertically divided into three areas, and divided-image signals are respectively taken as 77, 78 and 79 in the order from the left-hand area of the original image, and the divided-image signals 77, 78 and 79 are respectively introduced into encoders 62, 63 and 64. Each of the encoders 62 to 64 receives a size of the divided frame and an encoding condition from the division control unit 75 to encode the inputted divided-image signal, and then outputs as a code string 80, 81 or 82 to a code string synthesizer 74. Subsequently, the division control unit 75 outputs a code string synthesis signal which informs the code string synthesizer 74 of the frame division method according to the instruction given to the image divider 61, and in accordance with this code string synthesis signal, the code string synthesizer 74 connects the code strings from the decoders 62 to 64 in a specific sequence to produce the same syntax as that in the case of no frame division, thus finally outputting a synthetic or combined code string.
In the foregoing Japanese Unexamined Patent Publication No. 7-298263, in accordance with the instruction from a division control unit, an image signal originally corresponding to one image is divided into division image signals corresponding to a plurality of areas which in turn, are separately encoded in a plurality of encoders according to MPEG (Moving Picture Experts Group), and the code strings outputted from the plurality of encoders are arranged to produce the original syntax.
At this time, each of the encoders receives the size of the divided frame and the encoding condition to know the image signal, i.e., the area of the original image the encoder itself encodes, in other words, the image area in which the code string outputted by the encoder itself is finally displayed, before the MPEG encoding processing.
However, in the case of displaying reduced moving images (each of which will be referred hereinafter to as a thumbnail moving image) in arbitrary image areas of a multi-image raster (entire frame for multiple images), the displaying position is unknowable at the time that the MPEG encoding processing is conducted of each thumbnail moving image.
For this reason, at the time of combining or integrating multiple images, an operation to designate a displaying position is needed for each of thumbnail moving images encoded (which will be referred hereinafter to as an encoded thumbnail moving image).
Meanwhile, for example, in the MPEG standard (for reference, a portion of a syntax of a video stream in MPEG is shown in FIGS. 52A to 52D), a horizontal position at which a slice is displayed when decoded and reproduced is expressed by a variable-length code bit macroblock_address_increment (which will be referred hereinafter to as MB_Addr_Inc) of a leading macroblock in that slice. If making a change of the value of an MB_Addr_Inc section for the purpose of the disposition of the encoded thumbnail moving image at an arbitrary position, the number of bits of the MB_Addr_Inc section forming a variable-length code varies. Consequently, (in the case of merely applying to MPEG as the processing to designate a displaying position for the encoded thumbnail moving image,) at the combination or integration of moving images in a multi-image raster, a bit shift operation takes place by the bit number variation with respect to all the code strings following the MB_Addr_Inc section, which causes a large amount of calculation.
FIGS. 53A to 53C are illustrations available for describing this event. In the case that an encoded thumbnail moving image is disposed at a right and lower section of a multi-image raster comprising two divisions (two columns) in the vertical direction and two divisions (two rows) in the horizontal direction as shown in FIG. 53A, the code string of the encoded thumbnail moving image A varies as shown in FIG. 53B. When viewing the entire moving image, the vertical position of a slice is specified with the last 1 byte in a slice_start_code section.
On the other hand, the horizontal position of the slice is designated with the MB_Addr_Inc as mentioned above. Although the horizontal position of the slice varies from 1 to 5 due to the combination, if representing as a variable-length code, MB_Addr_Inc=1 signifies "1" being 1 bit, and if expressing as a variable-length code, MB_Addr_Inc=5 signifies "0010" being 4 bits, and hence, after the combination or composition, the bit string extends by 3 bit extra. Accordingly, as shown in FIGS. 53B and 53C, the code strings succeeding MB_Addr_Inc after the combination are located in a state of being shifted by the additionally increased 3 bits. Usually, computers or storage units for common use in the computers gain access in units of bytes. For this reason, as shown in FIG. 53C, all the code strings succeeding MB_Addr_Inc of the leading macroblock require the bit operation.
Furthermore, in the same MPEG standard, there is no guarantee for the code strings of the adjacent macroblocks to be separated (byte-aligned) in units of bytes, while as a solitary instance, the byte alignment is accomplished by next_start_code ( ) in FIGS. 52A to 52D after the code string of the last macroblock of a group of macroblocks constituting a slice. When the macroblocks arranged in the aiming order for the location of an encoded thumbnail moving image at an arbitrary position, the inter-macroblock connection crossing a byte border takes place, so that difficulty is experienced in decoding the code strings into a correct image.
Still further, in terms of the code string in the last macroblock of a slice, the byte alignment is made with padding bits as mentioned above. In the case that another macroblock code string is connected succeedingly to the macroblock code string which contains added padding bits, a decoder decodes code strings on the assumption that there exists no padding bit in the code strings of the adjacent macroblocks except that the code strings form the last of the slice. Thus, the decoder can not achieve the decoding into a correct image by the padding bits in the code strings.
FIGS. 54A to 54C are illustrations available for explaining this situation. In this instance, encoded thumbnail moving images A and B shown in FIG. 54A are combined into a multi-image raster comprising one divisions (one row) in the vertical direction and two divisions (two columns) in the horizontal direction. In the case that this combination processing is realized in a manner that macroblocks organizing each of the encoded thumbnail moving images A and B are disposed at arbitrary positions within the multi-image raster, the disposition of the respective macroblocks within the multi-image raster is as shown in FIG. 54B, and as shown in the lowest portion of FIG. 54C, the arrangement of the code strings thereof is such that the code string corresponding to the slice header section of the slice M1 first appears, and then, is followed by the succeeding macroblock code strings in the order of the macroblocks A1, A2, B1 and B2.
At this time, assuming that the last (end) macroblock A2 of the slice A1 is byte-aligned by padding bits, the code string of the slice M1 goes into a state in which a padding bit string is interposed in between the macroblocks A2 and B1. The result is that difficulty is encountered in correctly decoding an image.